CERIAS Tech Report 2006-52 WATERMARK AND DATA HIDING EVALUATION: THE DEVELOPMENT OF A STATISTICAL ANALYSIS FRAMEWORK

Kim, Hyung Cook. Ph.D., Purdue University, May, 2006. Watermark and Data Hiding Evaluation: The Development of a Statistical Analysis Framework. Major Professor: Edward J. Delp. Digital watermarking is the practice of hiding a message in an image, audio, video or other digital media elements. Since the late 1990’s, there has been an explosion in the number of digital watermarking algorithms published. But there are few widely accepted tools and metrics that can be used to validate the performance claims being asserted by members of the research community. Robust image watermarks are watermarks designed to survive attacks including signal processing operations and spatial transformations. To evaluate robust watermarks, we need to evaluate how attacks affect the fidelity of an image. The mean square error (MSE) is the most popular metric to measure fidelity. MSE, as it is, cannot measure the fidelity for images that went through geometric attacks such as rotation, pixel loss attacks such as cropping, or valumetric attacks such as gamma correction. We take the approach of evaluating attacks using MSE by compensating valumetric, pixel loss, and geometric attacks using conditional mean, error concealment, and motion compensation, respectively. Robust watermarks are evaluated in terms of fidelity and robustness. To measure robustness, bit error rate, message error rate and the receiver operating characteristic (ROC) of the watermark decoder and detector are currently used in the literature. We extend this framework by adopting reliability testing. We define reliability as the probability that a watermarking algorithm will correctly detect or decode a watermark for a specified fidelity requirement under a given set of attacks and images. We evaluate three watermarking algorithms in terms of quality (fidelity), load (wa-